(19) 



J 



Europais h s Patentamt 
Eur op an Pat nt Offic 
Office europeen des brevets 



(12) 



(43) DatG of publication: 

07.02.1996 Bulletin 1996/06 

(21) Application number: 95304268.6 

(22) Oateof filing: 20.06.1995 



(11) EP 0 696 017 A2 

EUROPEAN PATENT APPLICATION 

(51) int CI.6: G06T 3/40 



(84) 


Designated Contracting States: 


(72) 


Inventor: Yen, Jonatiian 




DE FR GB IT 




San Jose CA 95129 (US) 


(30) 


Priority: 05.08.1994 US 286561 


(74) 


Representative: Jehan, Robert et al 








London WC2E 7PB (GB) 


(71) 


Applicant: Hewlett-Packard Company 








Palo Alto, California 94304 (US) 







CM 
< 



CO 

o> 



(54) Binary image scaling 

(57) A piecewise polynomial interpolation scheme 
treats irriages as three-dimensional data in which the X 
and Y coordinates are the input image dimensions, and 
the Z coordinate is the intensity of the image. The 
three-dimensional data set (100) is fitted by a surface 
(120), and a resampling process (133) on the fitting sur- 
face provides interpolative data. A thresholding process 
( 1 40) applied on these interpolative data produces a final 
image output (150). Based on the interpolating scheme 
(140), each output pixel Is a weighted average of its 
neighbouring pixels, with weights determined by the type 
of the interpolant, its degree, and the desired scaling fac- 
tor. The weights may be pre-calculated for fixed scaling 
factors, such that the convolution is accomplished by ta- 
ble lookup. Additionally, the resampling process (130) 
may include a phase shifting to realign said sampling lo- 
cation with respect to an input innage. 
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The present invention relates to scaling binary images by piecewise polynomial in^^ 

When scaling a binary image, it Is necessary to resolve the discrepancy between the input resolution and the output 
5 resolution. For example, in facsimile Imaging the original images are transmitted at a standard resolution, typically 203 
dots per Inch ("dpi") in the row direction and 98 dpi in the column direction, and printed at the printer resolution, e.g. 300 
dpi, in both the row and column directions. Another example of binary imaging involves font scaling, for example when 
a 12 point font at 300 dpi Is to be used as a 24 point font and printed on a 600 dpi printer, the original Image must be 
scaled by a factor of four. Thus, the scaling factor can be an integer or a fractional number ^ 
^0 One approach to image scaling is to apply piecewise polynomial interpolation such as B-splines, as proposed in 

R V Klassen, R.H. Bartels, Using B-splines for Re-Sizing trnages. University of Waterloo, Department of Computer 
Science, Technical Report, 1986. However, such approach has not proven satisfactory. 

A quantization error problem embedded In all piecewise polynomial interpolants for scaling was reported in I.E. 
Abdou and K.Y Wong, Analysis of Linear Interpolation Schemes for Bi-Levef Image Applications, IBM J. Res. De- 
IS velop., V. 26, No. 6, pp. 667-680, November, 1 982. Thus, interpolation schemes give rise to a quantization error problem, 
such that the image stroke width is not preserved after image scaling. 

Other known approaches for scaling and smoothing images include pixel-replication, contour-tracing (see W. 
RutkowskI, Shape Completion, Computer Graphics and Image Processing, Vol. 9, pp. 89-101, 1979), cubic B-spline 
interpolation (see H.S. Hou, H.C. Andrews, Cubic Splines for Image Interpolation and Digital Filtering, IEEE Trahs- 
20 action on Acoustic, Speech and Signal Processing, Vol. ASSP-26. pp. 508-517), template based scaling (see R.A. 
Ulichney. D.E. Troxel, Scaling Binary Images with the Telescoping Template, IEEE Transactions on Pattern Analysis 
and Machine Intelligence, Vol. PAMM, No. 3, pp. 331 -335, 1 982). and extrapolative prediction (see C. Tung, Resolution 
Enhancement Technology in Hewlett Packard LaserJet Printers, \ST/SP\ ESympos'm^ Science 
and Technology, pp. 440-448, 1 993). 
25 The present invention seeks to provide improved image scaling. 

According to an aspect of the present invention, there is provided a method of binary image scaling as specified in 
claim 1 . 

According to another aspect of the present invention, there is provided scaling appairatus as specified in claim 11 . 

Preferably, the first and second coordinates are the X and Y coordinates and the third coordinate is the Z coordinate. 
30 It is possible with the preferred embodiment to provide a satisfactory and flexible scheme based on automatic numerical . 
computation for binary image scaling that is general enough for all integer and fractional scaling factors, while avoiding 
quantization probjems typically encountered in prior art approaches using piecewise polynomial interpolants. 

The described embodiment is based on a piecewise polynomial interpolation scheme. The scheme views images 
as three-dimensional data in \yhich the X and Y coordinates are the input inriage dimensions, and the Z coordinate, i.e. 
35 the height field, is the intensity of the original image. The three-dimensional data set is fitted by a surface. A resampling 
process on this fitting surface provides interpolative data. A thresholding process applied on these interpolative data 
produces an output, i.e. a final binary image. Based on the interpolation scheme, each output pixel is a result of blending 
its neighboring input pixels, i.e. it is a weighted average, with weights detemnined by the type of the Jnterpolant. its 
degree, and the scaling factor. For fixed scaling factors, the weights can be pre-calculated, and the convolution can be . 
40 accomplished by table lookup. 

An embodiment of the present invention is described below, by way of example only, with reference to the accom- 
panying drawings, in which: 

Fig.. 1 is a flow diagram showing ah exemplary embodiment of process path- 
os 

Fig. 2 shows a test character In 20 point, 300 dpi, Times Font; 

Fig. 3 is a wireframe representation of the input coefficients for the test character of Fig. 2; 

50 Fig. 4 shows a bi-linear tensor product B-spline surface defined by the input data of the test character of Fig. 2; 

Fig. 5 shows a bi-quadratic tensor product B-splihe surface defined by the input data of the test character of Fig. 2; 

Fig. 6 shows a bi-cubic tensor product B-spline surface defined by the input data of the test character of Fig. 2; 
ss • 

Fig. 7 shows a bi-linear interpolation of the test character of Fig. 2; 
Fig. 8 shows a bi-quadratic interpolation of the t st character of Fig. 2; 
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Fig. 9 shows a bi-cubic interpolation of th test charact r of Fig. 2; 
Fig. 10 shows an undershot quantization error. 
Fig. 11 shows an overshot quantization erron 
Fig. 12 shows the impulse response of a cubic B-spline; 

Fig. 1 3 shows the impulse response function of a cubic B-spline sampled at 0, 1/3, and 2/3; 
Fig. 14 shows the impulse response of a quadratic B-spline; 

Fig. 15 shows the impulse response function of a quadratic B-spline sampled at 0, 1/3. and 2/3; ^ 
t5 Fig. 16 shows a phase shift by one half of the spacing during sampling to compensate for phase distortion; 

Fig. 17 shows an image that results from the phase shift of Fig. 16; 

Fig. 18 shows the impulse response of a quadratic B-spline inteipolant after sampling at 0 and 1/2; 

Fig. 19 shows the impulse response of a cubic B-spline interpolant after sampling at 0 and 1/2; 

Fig. 20 shows quantization error for a bi-quadratic B-spline interpolant after sampling at 0 and 1/2, as shown in Fig. 
18; 

25 . 

Fig. 21 shows quantization error for a bi-cubic B-spline interpolant after sampling at 0 and 1/2, as shown in Fig. 19; 

Fig. 22 shows the impulse response of a quadratic Beta-spline; 

30 Fig. 23 shows an image that results from interpolating by a bi-quadratic Beta-spline; 

Fig. 24 shows the irnpulse response of a cubic Beta-spline; 

Fig. 25 shows an image that results from interpolating by a bi-cubic Beta-spline; ^ 

35 

Fig. 26 shows an image that results from interpolating by a shifted bi-quadratic B-spline; 
Fig. 27 shows an image that results from interpolating by a shifted bi-cubic B-spline; 
40 Fig. 28 shows a portion of a Slerexe letter in standard resolution; 

Fig. 29 shows a portion of a Slerexe letter in fine resolution; 
Fig. 30 shows a character cropped from a Slerexe letter; 

45 

Fig. 31 shows an image that results from applying a bi-cubic B-spline surface as an interpolant for scaling horizontally 
and vertically by a factor of 3.0 in fine resolution; 

Fig. 32 shows the result of applying a quadratic-by-quartic tensor product B-spline surface as an interpolant for 
50 scaling horizontally by a factor of 3.0 and vertically by a factor of 1 .5 in standard resolution; 

Fig. 33 shows a test character in the original 1 2 point font at 300 dpi; 

Fig. 34 shows the test character of Fig. 33 scaled two times by RET; and 

Fig. 35 shows the test character of Fig. 33 scaled by bi-septimal B-splines.. 

The xemplary embodiment is based on a piecewis polynomial interpolation scheme (see J. Yen, Binary image 
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Scaling by Piecewise Polynomial Interpolation, HPL Technical Report, March, 1994), It applies a surface based ap- 
proach to image scaling, as opposed to prior art contour-tracing schemes. 

Image scaling described herein is not limited to cubic transforms, as described in H,S. Hou, H.C. Andrews, Cubic 
Splines for Image Interpolation and Digital Filtering, IEEE Transactbn on Acoustic, Speech and Signal Processing, 
5 Vol ASSP-26, pp. 508-517. However, the exemplary embodiment does share some features with the template based 
approaches of R.A. Ulichney, D.E. Troxel, Scaling Binary Images with the Telescoping Template, IEEE Transactions 
on Pattern Analysis and Machine Intelligence, Vol. PAMI-4, No. 3, pp. 331-335. 1982; and C. Tung, Resolution En- 
hancement Technology in Hewlett Packard LaserJet Printers, IST/SPIE Symposium on Electronic Imaging: Science 
' and Technology, pp. 440-448.. 1993. However, it differs from such prior art approaches because it derives templates 
■ 10 mathematically from surface geometry rather than by interpretation or extrapolation by analysis of each individual case. 
Therefore the templates herein may be numerically fine-tuned for print quality control, as may be required, or as are 
consistent with, various selected design parameters. 

Fig. 1 is a flow diagram showing a process path for an exemplary embodirheht which provides a scheme that treats 
binary images as three-dimensional data in which the X and Y coordinates are the input image (110) dimensions, and 
'5 the Z coordinate, i.e. the height field, is the intensity of the original image. This three-dimensional data set is fitted by a 
surtace (120). A. resampling process (130) on the fitting surface provides interpolative data. A thresholding process 
(l40) is applied to the Interpolative data to produce an output binary image (150). Based on this interpolation scheme, 
each output pixel is a result of blending its neighboring input pixels, i.e. each output pixel is a weighted average of its 
neighboring pixels, with weights determined by the type of the interpolant, its degree, and the desired scaling factor. In 
20 one embodiment the weights can be pre-calculated for fixed scaling factors, such that the convolution can be accom- 
plished by table lookup. 



An Interpolation Method. 



2S The exemplary embodiment provides a scheme that employs a piecewise polynomial, for example, B-sptines, as 

the fitting interpolant to the original data (see W. J. Gordon and R. F. Riesenfeld, B-spline Cun/es and.Surfaces in R. 
. E. Barnhill and R. F. Riesenfeld (eds.), Computer Aided Geometric Design, Academic Press, New Yoirk: 1974, pp. 
95-126). ... 
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r(0 = 2L^/.*(0Pi (1) 



is a uniform polynomial B-spline curve in parameter te (0.1). where Bj i^ (t) are the spline basis, of order k (degree /c-1 ), 
and py are the input coefficients. In product forrn, 

55 Y(0= BP . (2). 

where 

. * ^=[So,;c(OB, ,,(0.-B^,;^(m : . (3) 

and 

'P^lPoP^^^Pk^y• (4) 



Example 1 

A cubic uniform polynomial B-spline curve, 

7(0 = [^04(0 S,.4(0 82.4(0 S3,4(0][Po Pi PaPal^' (S) 
is an inner product of four input coefficients p/s and the cubic spline basis Bi4(tys evaluated at / between [0, 1). 

Marsden's Identity. 



It can be proven (see L. Schumaker, Spline Functions: Basic Theory . Wiley, New York, 1 981 ) that for each spline 
basis of order /c, 

ss B=lB^Jt)B,Jt)../B^.,Jt)] . (6) 

there exists a basis translation matrix {k by ^)Mj^ , such that 
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where is the pow r basis, 



Since 7(0 = BP, therefore, 



B=TM„ 



|f* ' ... M). 



7(0 = TM^P. 



(7) 

(8) 
(9) 
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Example 2 

Given input coefficients, pg, p,, pg, and p^, a cubic uniform polynomial B-spline curve f is defined as 

7(0 = [fV f nMJiPa P, P2 P3]./e[0.1) 

where the basis translation matrix is 



(10) 
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(11) 



(see J. D. Foley, A. van Dam, S. K. Feiner. J. R Hughes, Computer Graphics. Principles and Practice . Second Edition, 
Addison Wesley. 1990 ). 

The basis translation matrices for other degrees, from linear through septimal, are described below. For nnore than 
k order input coefficients, every k consecutive coefficients define one piece of the curve segment such that the total 
curve consists of these curve segments and is thus called a piecewise B-spline curve. For a piecewise uniform poly- 
nomial B-spline curve, all the curve segments are connected smoothly, i.e. with C^"^ continuity at segment joints. 

A surface 
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(12) 
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is a uniform polynomial B-spline tensor product surface in parameters ue(d.1 ) and i/e[0.1 ), where: 
Bs are the spline basis, of order /c„ in direction U; 
As are the spline basis, of order k^ in direction V, and 
Pf jS are the two-dimensional input coefficients. 

Thus. s{u, V) = BPA''". similar to the curve case, A = UM^^ , A = WW^ , and A^ = where U is the power basis 



in u ; V is the power basis in v, and Ms are the basis translation matrices. 
Therefore. 

s(u,v)= UMf^ P^l^^- 



(13) 
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Example 3 



A bt-cubic uniform polynomial B-spline tensor product surface 



(14) 
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. ^ where A4j is the basis translation matrix of order 4, as shown 
Example 4 

5 Fjg.,2 shows a test character in 20 point, 300 dpi. Times Font. Fig. 3 is a wireframe representation of the input 

coefficients for the test character of Fig. 2. 

Fig. 4 shows a bi-linear tensor product B-spline surface defined by the input data of the test character of Fig. 2; Fig. 
. 5 shows a bi-quadratic tensor product B-spline surface defined by the input data of the test character of Fig. 2; and Fig. 
■ 6 shows a bi-cubic tensor product B-spline surface defined by the input data of the test character of Fig. 2. 
. , ' Fig. 7 shows a bi-linear interpolation of the test character of Fig. 2; Fig. 8 shows a bi-quadratic interpolation of the 
test character of Fig. 2; and Fig. 9 shows a bi-cubic interpolation of the test character of Fig. 2. In each of the foregoing 
figures, The letter "W" is shown as a series of steps in which the lower right hand letter includes missing portions of the . 
letter after scaling, indicated by sniall solid squares, and.added portions of the letter aftei- scaling^ indicated by small 
hollow squares. Notice that the result of bi-linear interpolation appears to be blocky, while the results of higher order 
^5 interpolation are usually smoother. . . / . • - 

B-splines As Convolution Kernels. 

Once the order of the interpolating surface is determined, the resampling process on the fitting surface evaluates 
20 the piece wise polynomial surface al a set of appropriate locations, one piece at a lime. 

For scaling by a factor of Min the L/direction, the evaluation locations are selected between [0, 1 ) with even spacing, 
usually ' . 

nl ^ Ml 
'MM' " A/f . 

25 in the U direction. 

For scaling by a factor of N in the V/directioh, the evaluation locations are selected between [0, 1 ) with even spacing, 
usually 

. ' 2 2 . ' 

' N'N' "' N 

30 in the V direction. : • 

Since each piece of the surface is defined by the kfj by consecutive input coefficients, resampling involves eval- 
uating 

/ X iiiLM oamTiJ rs 1 2 M-1 ^ 1 2 A/-1 . 

s(u.v)=UM^PM,V:u = 0,j^,j^...-i^,v = 0.j^.-....-^ (15) 

-35' T T 

For each known {Uq,Vq) pair, UM^^ and V can be pre-calculated as vectors, ^ and 0, respectively. A k^J by 
matrix, . called the Kerne/ Mafr/x, can be constructed by multiplying ^ and 0, A^^^ 

The coefficient matrix P is also Af^ by k^ The sum of the entry-by-entry product of Pand K is the result of the surface 
evaluated at u^ and Vq. 

s{Uo.yo)^^PijKu\ ; (17) 

(/.y) 

The kemel matrix is constructed for each {u,v) pair. 

Finally, a threshold value is selected to connpare against the numeric values, s(u,v)'s. to obtain the binary data of 
the output pixels. 

Example 5 

To scale an image by a factor of 3 with a bi-cubic uniform piecewise B-spline tensor product surface, s(u,v)i for each 
^ 4 by 4 input pixels, e.g. 
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0 110 
0 0 10 
0 110 
0 111 

10 a kernel matrix is calculated for each of the following 3 by 3 output pixels: 



(18) 
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(0.0) s[o.l] s(o.|) 



(19) 



For Instance, for the upper left comer of the 9 output pixels, s(uo, vq) , where Uq = 0.0 and vq = 0.0, U ifi u^ .0)= 
[0.0 0 0 0:01.0], and 



-1 1 

6 2 

1 £ 

6 3 



2 2 

"2 6 



Therefore. 
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16 .3 6 J 



(20) 
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Similarly, 
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(22) 



and 
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10 
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(23) 



The kernel matrix for s (0.0,0.0) can be calculated by multiplying entries of (UM) with (M^\/^, i.e. 
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(24) 



Finally, the sum of the entry-to-entry product of P and K, 



s(o.o.o.o) = 2^A,=^. 



(25) 
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is a numeric value representing the point on the surface after resampling. The final quantization step compares this 
numeric value against a pre-selected threshold value to produce a binary value representing the upper-left comisr of 
the output binary image. 

The kernels for the other s(u, v)'s are listed below. 

B-spline Characteristics. 

There are many advantageous mathematical characteristics of B-splines. Some of the advantages of using B-splines 
as interpolants for binary image scaling are listed below: 

Continuum. The fitting surface, i.e. a B-spline surface, is a continuum (X>vering the entire Input data. Accordingly, 
the resampling process always produces at least a few points on the surface, regardless of the arbitrariness of the 
scaling factors. Thus, this interpolating method is applicable for any arbitrary scaling factor. 

Convex Hull Property. A B-spline tensor product surface is always contained in the convex hull formed by Its input 
data mesh. Thus, all the output data produced from the resampling process are always bounded by the Input data. 

Continuity. A B-spline tensor product surface is C^-^ continuous in each parametric direction, where /c is the order. 
Thus, the interpolated data are smooth. If a higher order Interpolant is used, the edges of the output image are smoother 

Locality. Each point oh the B-spline tensor product surface depends only on a finite number of input data points, k^J 
by where ks are the orders of the surface. Thus, the convolution kernel size of each output pixel is no more than /c^ 
by/c^ 

Separability. The surface evaluation process of a B-spline tensor product surface may be totally independent from 
one parametric direction lo the other. Accordingly, this interpolation scheme works for different scaling factors in the two 
orthogonal directions. In addition, the order of one direction is totally independent from the order of the other direction. 
Thus, the selection of the interpolant orders may be as arbitrary as required. 

Symmetry. AH the B-spline basic functions are symmetric and their. impulse response functions are also symmetric. 

Computational Complexity. 

The complexity of the schenrie herein described is linear in the input size because it has a fixed number of operations 
per pixel. The scheme may be thought of as constant filtering with a set of k^J by /c^ convolution kernels, where ks are 
the orders, i.e. degrees plus one, of the fitting surface. The number of the filters is at most M* where A/f and A/ are 
the scaling factors in the horizontal and vertical directions, respectively. 
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The filtering process of the scheme d scribed h rein depends only on the scaling factor, the orders of th fitting 
surface, and the threshold value. Therefore, after the filter parameters are finalized, a template scheme may easily b 
implemented by pattern matching and table lookup. 

k k 

s The lookup table size may be up to about 2 " ^ *M^Nblis. However, much reduction in table size may be achieved 

by using the mathematical properties of B-splines. For example, when scaling by a factor of 3 with a bi-cubic uniform 
piecewise B-spline tensor product surface, the upper bound of the space complexity Is 2^*^ * 3*3 bits. That is. for each 
of the 9 output pixels, there is a table of 16 entries, as described below. 

Locality. Upon close examination, it can be seen that not alt of the 9 output pixels have 1 6 entries. For example, the 
convolution kernels of many output pixels have all O's for the rightmost column or the bottommost row. The complexity 
of the, lookup tables can therefore be reduced to 2®+2'' 2*4+2^^*4 bits. 

Symmetry. If one looks even more closely, one can also discover that there is some symmetry in the numeric values 
in the convolution kernels. For example, the kemels 

IS 0,1 o.| ^.0 1.0 
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K^^.K^,K^\k^ (26) 



have similar rows or columns except the numbers are row-permutations or column-permutations of each other Similar 
relationships hold for 

11 12 12 12 

■^/<^ ■^ K^'^. (27) 

thus, the space complexity of the lookup tables can be further reduced to 2®+2^2+2i6 bits, if some processing 
capability is provided. 

Dominancy. Some entries of the convolution kemels are more dominant with regard to the resulting output than are 
other entries. A final thresholding process is applied for binary Image scaling. Many entries of the kemels become 
insignificant because of this quantization process. For example, if the threshold value is chosen as 16/32 for the above 
case; then in kernel of s (0.0), those entries with value 1/36 are effectively "dont-cares" because they never contribute 
to the final result in any significant way. For practical purposes, the various implemeritations described may be optimized 
by keeping only the most dominant entries of kernels. In this way, one may fine tune the process for each specific 
applicatk>n by balancing between a practical table size and a desired print quality. ^ 

In summary, the lookup table size can be significantly reduced depending on the design constraints in memory 
limitation arid processor speed. 

A Quantization Error Problem. 



For scaling binary images, it is important to preserve the stroke width. That is, when a single pixel line is scaled by 
a factor of N, the output pixel line must have a width N. Sometimes the stroke width is not preserved if errors are produced' 
from the quantization step of the scaling algorithm, for example when a single pixel line is scaled by a factor of 3 by 
means of a bi-quadratic B-spline ihterpolant. 
: Fig. 10 shows an undershot error in quantization with threshold value H= 0.5; and Fig. 11 shows an overshot error 
in quantization with threshold value H= 0.5+1 Oer^. No matter how one adjusts the threshold value, the scaled image 
either consistently has an extra pixel line, or it has one less pixel line than the pixel replication. Similar errors occur when 
a. single pixel line is to be scaled by a factor of 2 by means of a bi-cubIc B-spiine interpolant. 

The quantization error problenn was reported in I.E. Abdbu and K.Y. Wong, Analysis of Linear Interpolation 
Schemes for Bi-Level Image Applications, IBM J. Res. Develop., V. 26, No. 6, pp. 667-680, November, 1982, and was 
believed to be caused by the lack of symmetry of all the even degree polynomials. However, as shown above, the 
quantization error occurs regardless of the degree of the interpolant. 

For example. Fig. 12 shows the impulse response of a cubic B-splihe. while Fig. 13 shows the response function 
sampled at 0, 1/3, and 2/3. The peak of the response function, representing the input data point, is included in the 
resampled data. In contrast thereto (see Figs. 14 and 15), when scaling by 3 with a quadratic B-splihe interpolant, the 
peak of the response function is not included in the resampled data. This results because of sampling the interpolant 
at the locations, 

*A/* • N 

^ . This phenomenon has been described as the phase distortion property of all the v n degree polynomials (s e I.E.- 
Abdou and K.Y. Wong, Analysis of Linear Interpolation Schemes for Bi-Level Image Applications, IBf^ J. Res. D - 
velop.. V. 26, No. 6. pp. 667-680, Novemb r. 1982). 
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As shown in Fig. 1 6, the phase could be shifted by one half of the sampling spacing, e.g. by 1 /6, to compensate for 
such phase distortion. Fig. 1 7 shows the result of such an adjustment, having a threshold value H = 0.56. It can be seen 
that for odd scaling factors, when compared to the situations shown in Figs. 10 and 11 . B-spline interpolants, whether 
of even degree or of odd degree, do not suffer quantization errors If the phase is not distorted. 

s However, for even-scaling factors, all B-spllne interpolants; regardless of whether they are of even degree or of odd 

(jegree, are subject to a quantization error, even if phase is not distorted. For example, Fig. 18 shows the impulse 
response of a quadratic B-spline interpolant after sampled at 0 and 1/2, i.e. a scaling factor of 2; and Fig. 1 9 shows the 
impulse response of a cubic B-spline interpolant after sampled at 0 and 1/2. In both cases, there is no phase distortion. 
Nevertheless, quantization errors occur in both cases, as shown in Figs. 20 and 21. This quantization error is caused 

10 by a lack of symmetry of the interpolant. That is, in the absence of phase distortion, the peak of the impulse response 
is always sampled, while the rest of the sampled data points are always symmetric. Hence, when the thresholding step 
takes place, the result always contains an odd number of output pixels. 

One aspect solves the quantization error problem caused by a lack of the symmetry of the impulse response of 
B-spline interpolant when even scaling factors are applied. In one embodiment one aspect of this solution enlarges the 

IS search space from the space spanned by B-splines. which' is equivalent to the space containing all the piecewise pol- 
ynomials, to the space spanned by the Beta-spline, which is a more generalized form of splines ( see A. Lempel and G. 
Seroussi. Sysfemar/c derivation of spline bases. Computer Aided Geometric Design, 9 (1992), pp. 349-363; and.G. 
SeroussI and B: Barsky, An explicit derivation of discretely shaped Beta-spline basis functions of arbitrary order, 
Mathennatical Methods in Computer Aided Geometric Design II, T. Lyche and Larry Schumaker (eds.): pp. 567-584, 

20 1992). 

The use of enlarged search space is derived in part from the fact that the impulse response .of a Beta-spline is 
usually not symmetric. Fig. 22 shows the impulse response of a quadratic Beta-spline with Pi = 1 .3, and Fig. 23 shows 
the result of using this interpolant with threshold value H = 0,56; while Fig. 24 shows the impulse response of a cubic 
Beta-spline with = 2;0, p2 = -0 5, and Fig. 25 shows the result of using this interpolant with the threshold value H = 
25 0.52. Unfortunately, the foregoing solutions to the quantization error problem still produce a scaled output Image that is 
missing at least one pixel in horizontal run length. 

A solution to the quantization error problem overcomes the symmetry of the impulse response by shifting the phase 
of the basis function. Fig. 26 shows the result of interpolation by a bi-quadratic B-spline with phase shift 6 = 0.1 and 
threshold value H = 0.56, while Fig. 27 shows the result of interpolation by a cubic B-spline with phase shift 5 = 0.1 and 
30 threshold value H = 0.56. The actual amount of phase shifting used is a function of the application. 

/Vnother solution to the quantization error problem combines the two techniques described above, i.e. it uses a 
phase shifted, Beta-spline interpolant. 

Applications. , 

The scheme described herein was applied to two major application areas, e.g. facsimile imaging and font scaling. 
It should be appreciated that the scheme is not limited to these two example applicatbns. 

For facsimile imaging in standard mode, the input images are transmitted in standard facsimile resolutbns of 203 
dpi horizontally and 98 dpi vertically. Fig. 28 shows a portion of the Slerexe letter in standard resolution. In fine mode, 
40 the vertical resolution is 196 dpi; Fig. 29 shows a portion of the Slerexe letter in fine resolution. 

At the receiving endof the facsimile transmission, printer resolutbn is typically 300 dpi both ways. 600 dpi horizontally 
and 300 dpi vertically, or 600 dpi both ways. Therefore, for facsimile imaging the scaling factors are usually fixed numbers. 
However, because of complications associated with the physical constraints in printable margins of the facsimile printing 
mechanism, the scaling factor is not actually 300/203 or 300/98. Instead, the scaling factors are rounded off to 3/2 or 
45 3/1 (/.a 1.5 or 3.0) by padding with extra pixels to fill in the differences encountered in various firmware implementations.. 

In addition to binary image scaling, the scheme may also be used for image smoothing to remove the aliasing 
artifacts, as is required in facsiimile imaging. Fig. 30 shows a character cropped from the Slerexe letter. The aliasing 
artifacts shown in the figure include staircases, edge licks, and stray dots. As a result of applying a bi-cubic tensor 
product B-spline surface as an interpolant, the staircases are smoothed, the number of stray dots is reduced, and edge 
50 ticks are increased in run length and lowered in height. In fact, stray dots may be detected and totally removed by the 
invention, if so desired. ^ 

Fig. 31 shows the results of applying a bi-cubic B-spline surface as an interpdant for scaling horizontally and ver- 
tically by a factor of 3.0 in fine resolution. If it is necessary or desirable to compensate for the 2:1 aspect ratio used in 
standard resolution, a different order in horizontal direction may be chosen from that of the vertical direction of the fitting 
55 surfac . Fig. 32 shows the result of applying a quadratic-by-quartic tensor product B-spline surface as an interpolant for 
scaling horizontally by a factor of 1 .5 and vertically by a factor of 3.0 in standard resolution. 

The scheme may also be used to scale English character fonts. For example, to scale a 12 point font in 300 dpi to 
a 24 point font for printing on a 600 dpi printer requires scaling by a factor of 4 in both th horizontal and the vertical 
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directions. Fig. 33 shows a test character In the original 12 point font in 300 dpi. 

The test character of Fig. 33 may be scaled by twice applying RET (Resolution Enhanc m nt Technology), which 
uses a set of diamond shaped templates (see C. Tung. Resolution Enhancement Technology in Hewlett Packard 
LaserJet Printers, IST/SPIE Symposium on Electronic Imaging: Science and Technology, pp. 440-448. 1993). Fig. 34 
5 shows the results obtained using this approach. In the figure, it can be seen that artifacts 34 have been introduced along 
the inside curved portion of the test character at lower right comer by this approach. 

One embodiment that was applied to the test character involved interpolating by a bi-septimal tensor product B-spline 
surface using a rectangular template. This approach removes the artifacts, as can be seen in Fig. 35. 

10 Basis Translation Matrices. 



For each spline basis of order /c 

^ = [BojnB,Jt)... B,,,Jt)l 
there exists a basis translation matrix (k by k) M^^ such that 



20 



2S 



where T^^ is the power basis. 



[t^'^ t^'^ ... M]. 



(28) 
(29) 
(30) 



Listed belovy are basis translation matrices for uniform polynomial B-splines of orders linear through septimal. 
Linear: . 



M2 



-[V 11 



(31) 



Quadratic: 



30 



3S 



Afe= 1/2 
(32) 



1 -2 1 
-2 2 0 
1 1 0 



Cubic: 



40 



4S 



(33) 



-1 3 

3 -6 

-3 0 

1 4 



-3 


1 


3 


0 


3 


0 


1 


0 



Quartic: 



so 



ss 
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Ms = 1/24 



10 



Quiiitic: 



1 -4 6 4. 1 

-4 12 -12 4 0 

6 -6 -6 6 0 

-4 -12 12 4 0 

11111 10 



(34) 



IS 



Mq= 1/120 



20 



-1 5 -10 10 -5 1 

5 -20 30 -2b 5 0 

-10 20 0 -20 10 0 

10 20 -60 20 10 0 

-5 -50 0 50 5 0 

1 26 66 26 1 0 



(35) 



Hexadic: 



2S 



30 



35 



40 



45 



SO 





* 1 


-6 


15 -20 15 


-6 


1" 










-6 


30 


-60 60 -30 


6 


0 










15 


-45 


30 30 -45 


15 


0 








Mr = 1/720 




-20 


-20 


160 -160 20 


20 


0 










15 


135 


-150 -150 135 


15 


0 










-6 


-150 


-240 240 150 


6 


0 










1 


57 


302 302 57 


1 


0_ 


■ 






(36) 


















Septimal: 




















■ -1 


,7 


-21 35 


-35 


21 


-7 


1" 




.7 


-42 


105 -140 " 


105 


-42 


7. 


0 




-21 


84 


-105 0 


105 


-84 


21 


0 




35 


0 


-315 560 


-315 


0 


35 


0 


Me = 1/5040 


-35 


-280 


665 0 


-665 


280 


35 


0 




21 


504 


315 -1680 


315 


504 


21 


0 




-7 


-392 


-1715 0 


1715 


392 


7 


0 




1 


120 


1191 2416 


1191 


120 


1 


0 
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55 Sample Convolution Kernels. 

Listed below are convolution kernels for scaling by a factor of 3 with a bi-cubic uniform polynomial B-spline tensor 
product surface. s(u,v). K*^^^ is the kernel for sampling s {u, v) at (Uq, Vq), 
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36 



1 4 10 

4 16 4 0 

14 10 

0 0 0 0 



(38) 



10 



IS 



972 



8 93 60 1 

32 372 240 4 

8 93 60 1 

0 0 0 0 



(39) - 



20 



25 



30 



.0.1 



K ' = 



972 



972 



1 60 93 8 

4 240 372 32 

1 60 93 8 

0 0 0 0 

8 32 8 0 

93 372 93 0 

60 240 60 0 

1 4 10 



(40) 



(41) 



3S 



40 



2 1 



26244 



64 744 480 8 

744 8649 5580 93 

480 5580 3600 60 

8 93 60 1 



(42) 
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26244 



8 480 74 64 ' 

93 5580 8649 744 

60 3600 5580 480 

1 60 93 8 



-1.0 



972 



1 4 10 

60 240 60 0 

93 372 93 0 

8 32 8 0 



(43) 



(44) 
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10 



IS 



^20 



2S 



30 



2 1 



2 2 





V 8 


93 


60 


1 ■ 


1 


480 


5580 


3600 


60 


26244 


744 


8649 


5580 


93 ■ 




.64 


744 


480 


8. 




" 1 


60 


93 


8 ■ 


1 


60 


3600 


5580 


480 


26244 


93 


5580 


8649 


744 * 




_8 


480 


74 


64 _ 



(45) 



(46) 



Accordingly, this example provides an image scaling method that is based on piecewise polynomial interpolation: 
It may be used for arbitrary or fixed scaling factors, and it may also be used to reduce or remove aliasing artifacts. 
The example described herein has the following advantages: 

It is a formal model that provides a systematic way of generating convolution kernels for image scaling that are 
useful for solving the quantization error problem that affects all polynomial interpolants. * 

It provides a scheme that determines numeric values that are used to control the weights of the neighboring input 
pixels. By adjusting the mathematical computations applied by the invention, the specific implementation may be fine 
tuned for the best print quality, subject to various design parameters, such as pen, ink. paper, carriage speed, and ROM 
size. ■ 

It provides a table lookup scheme suitable for hardware implementation using known hardware techniques. The 
lookup table size may be substantially reduced to meet the memory space constraints. 

The disclosures in United States patent application no 08/286,561 , from which this application claims; priority; and 
in the abstract accompanying this application are incorporated herein by reference. . ' 



Claims 
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2. 



4. 



A method of binary image scaling by piecewise polynomial interpolation, comprising the steps of: . 

modelling a binary image as three-dimensional data (X.Y,2) In which the first and second coordinates (X.Y) 
represent input image dimensions, and the third coordinate (2) represents the intensity of the original Image; 

fitting the three-dimensional data set by a surface interpolant; . . 

resampling said input binary image on the fitting surface to provide interpblatlve data; 

thresholding said interpolative data; and . 

outputting a scaled binary image. 

A method as in claim 1 , comprising the steps of: 

determining the type and the degree of the piecewise polynomial interpolant for surface fitting based on the 

image scaling factor. 

A method as in claim 1 or 2, wherein said interpolant is a B-spline tensor product surface or a Beta-spline tensor 
product surface. 

A method as in any one of claims 1 to 3. comprising the step of: 

phase shifting during said resampling step to realign said sampling location with respect to said scaling factor. 

A method as in claim 4, comprising the steps of: . 
determining the amount of phase shifting for reducing quantization error 

A method as In any preceding daim, comprising the step of: 

determining threshold values used to produce binary output pixels. 
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7. A methcxJ as in any preceding daim, comprising the steps of: 

determining values used to control weights of neighbouring input pixels; 

pre-calculating said weights for fixed scaling factors; and performing a convolution by table lookup. 
5 8. A method as in claim 7, comprising the step of fine tuning said table lookup. 

9. A method as in claim 7 or 8, comprising the steps of: 

reducing table lookup size by the locality detected in said convolution kernels; 
reducing table lookup size by the symmetry detected in said convolution kernels; and 
'9 reducing table lookup size by keeping only the most dominant entries of the convolution kernels. 

10. A method as in any one of claims 7 to 9, comprising the step of: 

manually modifying the entries of the lookup table, so as substantially to remove aliasing artifacts. 

'5 11. Scaling apparatus for binary image scaling by piecewise polynomial interpolation, comprising: 

an input module (110) for receiving a binary image as three-dimensional data (X, Y.Z) in which first and second 
coordinates (X.Y) represents input image dimensions, and the third coordinate (Z) represents the intensity of the 
original image; 

a surface fitting module (120) for fitting the three-dimensional data set by a surface; 
20 a resampling module (1 30) for resampling said input binary image on the fitting surface with weights determined 

by type of interpolant, its degree, and scaling factor, to provide interpolative data; 
a thresholding module (1 40) for thresholding said interpolative data; and 
an output module (150) for outputting a scaled binary image. 

12. Scaling apparatus as in claim 11. comprising: 

means for determining the type and degree of the piecewise polynomial Interpolant for surface fitting (120) 
based on the scaling factor. 

1 3. Scaling apparatus as in claim 11 or 12, wherein said interpolant is a B-spllne tensor product surface or a Beta-spline 
30 tensor product surface. " , 

14. Scaling apparatus as In any one of claims 11 to 1 3, comprising: 

means for phase shifting said binary input image to realign said sampling location with respect to said scaling 
factor. 

15. Scaling apparatus as in claim 14, comprising: 

means for determining the amount of phase shifting so as to reduce quaritization error. 

16. Scaling apparatus as in any one of claims 11 to 15. comprising: 

means for determining threshold values used to produce binary output pixels (1 40). 

17. Scaling apparatus as in any one of claims 11 to 16, comprising: 

means for determining values used to control weights of neighbouring input pixels; 

means for pre-calculating said weights for fixed scaling factors; and means for peilormtng a convolution by 
^ table lookup. 

18. Scaling apparatus as in claim 1 7, comprising means for fine tuning said table lookup. 



so 



ss 



19. Scaling apparatus as in claim 1 7 or 18, comprising: 

means for reducing table lookup size by the locality detected in said convolution kernels; 

means for reducing table kx)kup size by the symmetry detected in said convolution kernels; and 

means for reducing table lookup size by keeping only the most dominant entries of the convolution kernels. 

20. Scaling apparatus as in any one of claims 17 to 1 9, comprising: 

means for manually modifying the entries of the lookup tabi so as substantially to remov aliasing artifacts. 
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(54) Binary image scaling 

(57) A piecewise polynomial interpolation scheme 
treats images as three-dimensional data in which the X 
and Y coordinates are the input image dimensions, and 
the Z coordinate is the intensity of the image. The three- 
dimensional data set (100) is fitted by a surface (120). 
and a resampling process (130) on the fitting surface 
provides interpolative data. A thresholding process 
(140) applied on these interpolative data produces a fi- 
nal image output (150). Based on the interpolating 
scheme (140). each output pixel Is a weighted average 
of its neighbouring pixels, with weights determined, by 
the type of the interpolant, its degree, and the desired 
scaling factor. The weights may be p re-calculated for 
fixed scaling factors, such that the convolution is accom- 
plished by table lookup. Addrtronally, the resampling 
process (130) may include a phase shifting to realign 
said sampling location with respect to an input image. 
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